Stemming and Segmentation for Classical Tibetan

Tibetan is a monosyllabic language for which computerized language tools are largely lacking. We describe the development of a syllable stemmer for Tibetan. The stemmer is based on a set of rules that strive to identify the vowel, the core letter of the syllable, and then the other parts. We demonstrate the value of the stemmer with two applications: determining stem similarity of two syllables and word segmentation. Our stemmer is being made available as an open-source tool and word segmentation as a freely-available online tool. It is worthy of remark that a tongue which in its nature was monosyllabic, when written in the characters of a polysyllabic language like the Sanskrit, had necessarily to undergo some modification. Sarat Chandra Das, “Life of Sum-pa mkhan-po, also styled Ye-śes dpal-’byor, the author of Rehumig (Chronological Table)”, Journal of the Asiatic Society of Bengal (1889)